DOCtmSXT RESUME 



ED 334 486 



CG 023 501 



AUTHOR 
TITLE 



INSTITUTION 



SPONS AGENCY 
PUB DATE 
NO"!E 
PUB TYPE 



Reardon, Robert C; And others 

A comparative Analysis of the Impact of Two 

Generations of a Computer-Assisted Career Guidance 

System— SIGI and SIGX Plus; Technical Report No. 

7. 

Florida State Univ., Tallahassee. Center for the 
Study of Technology in counseling and career 
Development. 

Kellogg Foundation, Battle Creek, Mich. 
May 88 

35p.; For related reports, see CG 023 498-500. 
Reports - Research/Technical (143) — 
Tests/Evaluation Instruments (160) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



HP01/PC02 Plus Postage. 

Career Counseling; -Career Information Systems; 
College Students; Comparative Analysis; Computer 
Assisted Instruction; Higher Education 
Computer Assisted Career Guidance; -system of 
Interactive Guidance and Info Plus; "System of 
Interactive Guidance and Information 



ABSTRACT 

This study examined the comparative impact of two 
generations of a computer-assisted career guidance system. System of 
Interactive Guidance and Information (SIGI), and SIGI PLUS, on the 
career decidedness, vocational identity, and user perceptions of 64 
college students. The students were randomly assigned to either SIGI 
or SIGI PLUS. The occupational Alternatives Question, and My 
Vocational Situation were completed during pre- and post-treatment 
while the SIGI or SIGI PLUS Evaluation Form (a measure of user 
perceptions) was completed during post-treatmpnt . After the 
completion of initial post-testing, students ised the second system 
and completed the Comparative Computer Rating Form. Users rated SIGI 
and SIGI PLUS equally positive for their effectiveness in obtaining 
self and occupational information (analysis) , viable career options 
(synthesis), and in attractiveness of interaction with the computer 
(computer effect) . Persons with initial information needs rated both 
CACG systems significantly higher on the analysis scale than users 
without needs for information, students' level of vocational identity 
increased significantly as a result of using SIGI and SIGI PLUS. 
Students with low initial vocational identity were more likely to 
increase their vocational identity if they used SIGI PLUS, than if 
they used SIGI. SIGI and SIGI PLUS appear equally capable of 
satisfying users' needs for information. When the students were asked 
to indicate their overall preference for one of the two systems, 38 
percent preferred SIGI, while 62 percent preferred SIGI PLUS. The 
survey instrument and S'u references are attached. (Author/LLL) 
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Abstract 

A study was conducted to examine the comparative impact o£ 
two generations of a computer-assisted career guidance system, 
SIGI and SIGI PLUS, on the career decidedness, vocational 
identity, and user perceptions of 64 college students. Students 
from an introductory psychology class who expressed interest in 
an experiment that involved receiving career guidance services 
were randomly assigned to either SIGI or SIGI PLUS. The 
occupational Alternatives Question , and My vocational situation 

were completed pre- and post-treatment while the SI£X or SIS1 
PLUS Evaluation Form (a measure of user perceptions) was 
completed post-treatment. After the completion of initial post- 
testing, students then used the second system and completed the 
Comparative Computer Rating Form which involved a direct bipolar 
comparison of the two systems. A series of multivariate and 
univariate statistical analyses revealed that users rated SIGI 
and SIGI PLUS equally positive for their effectiveness in 
obtaining self and occupational information (Analysis) , viable 
career options (Synthesis) , and in attractiveness of interaction 
with the computer (Computer Effect) . Persons with initial 
information needs rated both CACG systems significantly higher 
(p< .05) on the Analysis scale than users without needs for 
information. Students' level of vocational identity increased 
significantly (p*.006) as a result of using SIGI and SIGI PLUS. 
Students with low initial vocational identity were more likely to 
increase their vocational identity if they used SIGI PLUS, than 
if they used SIGI. SIGI and SIGI PLUS appear equally capable of 
satisfying users' needs for information. Finally, when the 63 
students were asked to indicate their overall preference for one 
of thri two CACGS used, 24 or 38% preferred SIGI, while 39 or 62% 
preferred SIGI PLUS* Discussion of these results explored the 
differential impact of SIGI and SIGI PLUS and the equivalence of 
CACG systems. Specific implications for practice, system 
development, and future research are also provided. 
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Background 



Computer-assisted career guidance (CACG) and information 
systems have become a common resource in the provision of career 
counseling and guidance services (Chapman & Katz, 1983; McKinlay, 
1988; Sampson, Shahnasarian, & Reardon, 1987). Numerous studies 
have been conducted to determine software effectiveness within 
the context of various individual, group, and curricular-based 
intervention strategies (Cairo, 1983; Clyde, 1979; Parish, 
Rosenberg, & Wilkinson, 1979; Sampson, 1984) . These data have 
been used by practitioners to select software, by system 
developers to refine computer applications and develop new 
software, and by researchers as a basis for conducting future 
research. 

The rapid rate of software development, due partly to the 
ease with which software can be modified and the rapid changes in 
hardware capabilities and costs, has created an unprecedented 
problem for practitioners, system developers, and researchers. 
Traditional career guidance resources, e.g. assessment 
instruments, work sample kits, monographs, etc., were revised 
relatively infrequently due to the high cost of validating, 
manufacturing, and shipping new materials. Because software is 
leased on a yearly basis (and therefore replaced) and the cost of 
manufacturing and shipping computer disks is relatively low, 
revised or entirely new versions of existing software are 
produced more frequently. Most of the widely-used CACG systems, 
including CHOICES, CIS, DISCOVER, CIS, and SIGI PLUS, have 
evolved significantly from their initial production versions . 
While this capacity for relatively rapid change and anticipated 
improvement is an important advantage of computer software, 
problems can arise when an attempt is made to generalize across 
versions of a particular system or generalize about a particular 
system's performance over time. Major CACG system changes could 
in fact lessen a system's effectiveness, while professionals 
would continue to assume that prior research data was still 
valid. What is needed is a methodology for studying evolving 
versions of CACG systems so that: 1) professionals can have some 
preliminary ideas as to how a specific version of a CACG system 
will impact the career behavior of clients; and 2) developers and 
researchers can understand more about comparative effects. 

A case in point involves the System of Interactive Guidance 
and Information (SIGI) and SIGI PLUS. The original SIGI system 
was designed at the Educational Testing Service under the 
leadership of Martin Katz, with development beginning in 1971 and 
the first production version released in 1976. SIGI was designed 
to help students clarify their values, identify and explore 
occupational alternatives, receive and interpret relevant 
occupational and educational data, and master strategies for 
making informed and rational career decisions (Katz, 1973, 1980). 
The five SIGI subsystems include: 1) VALUES, 2) LOCATE, 3) 
COMPARE, 4) PLANNING, and 5) STRATEGY. A variety of theoretical 
(Katz, 1966, 1968, 1969, 1973) and empirical work (Katz & Norris, 
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1972; Katz, Norris, & Kirsh, 1969; Norris, 1977; Norris 6 
Chapman, 1976, Norris & Cochran, 1977) was completed as part of 
the system development process. 

SIGI has been effective in increasing career preparedness 
(Neumann, 1978) ; encouraging career search behavior (Davis & 
Dickson, 1980) ; improving career decision making skills (Chapman, 
Katz, Norris, & Pears, 1977; Cochran, Hoffman, strand, :■ warren, 
1977; Riesenberg, 1980); enhancing confidence in educational and 
vocational planning (Kapes, Borman, and Frazier, 1986; Tulley * 
Risser, 1976) ; improving attitude toward career planning and 
career exploration (Kapes, Borman, and Frazier, 1986) ; decreasing 
the level of undecidedness (Salters, 1984) and indecision (Kapes, 
Borman, and Frazier, 1986) ; providing valid career guidance 
services for students with a variety of learning styles (Pelsma, 
1982), and stimulating students' perceived needs for follow-up 
career counseling (Sampson & Stripling, 1979) . There have been 
mixed results in terms of the impact of SI6I on career maturity, 
with Pyle and stripling (1976) showing an increase while Devine 
(1975) and Fadden (1983) showing no improvement. In addition, 
Lotterhos (1980) found that SIGI did not significantly contribute 
to community college student persistence or academic achievement. 

A new version of SIGI entitled "SIGI PLUS" was designed 
under the leadership of Li la Norris at the Educational Testing 
Service, with development beginning in 1980 and the first 
production version released in 1985. The new system was 
developed in response to comments from users and counselors 
concerning the effectiveness of SIGI with a diverse group of 
individuals, especially adults. Like SIGI, SIGI PLUS is designed 
to facilitate rational career decision making. In comparison 
with SIGI, SIGI PLUS; 1) provides greater diversity of self- 
assessment options, 2) is more flexible in terms of user control 
of system functioning, 3) provides specific content material 
related to the needs of typical adult learners as well as 
traditional college-age students, 4) includes content related to 
the job search process, 5) provides for easier customization of 
local data, and 6) makes use of color graphics. SIGI PLUS has 
nine sections that include: INTRODUCTION , SELF-ASSESSMENT, 
SEARCH, INFORMATION, SKILLS, PREPARING, COPING, DECIDING, and 
NEXT STEPS. Katz (1984) provided a description of the initial 
design of the system. The basic assumptions and design features 
of the system are described by Norris, Shatkin, Schott, & Bennett 
(1985) . 

Field test data reported by Norris, Shatkin, Schott, & 
Bennett (1986, p. 49) indicated that users found t*e system, 
11 interesting, useful, and free from bias, its information 
plentiful and easy to use, and its writing style and vocabulary 
appropriate." Seeger (1988) found that use of SIGI PLUS resulted 
in improvement in college students' career development attitudes 
and knowledge. Hafer (1987) reported that SIGI PLUS was 
effective in reducing college students' career indecision. 
Buglione and DeVito (1986) found SIGI PLUS users to be 
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overwhelmingly positive in their perceptions of system 
effectiveness. Rogers (1984) reported that expert judges in 
adult learning and CACG, career guidance practitioners, and 
adults in career transition rated the design features of the Next 
Steps section of SIGI PLUS as theoretically consistent in terms 
of adult development and CACG. 

Evidence currently exists as to the effectiveness of SIGI in 
relation to a variety of career development outcome measures. 
While preliminary data does exist as to the effectiveness of SIGI 
PLUS, it is not clear at present if this nev system has a similar 
impact or performs as effectively as the preceding version. 

Purpose of the Study 

The purpose of this study is to demonstrate a methodology 
for analyzing the performance of two versions of a CACG system 
that can be used with a wide variety of computer applications in 
career guidance and to provide specific data on the comparative 
impact of SIGI and SIGI PLUS. 

Methodology 

Sample 

Sixty-four students from an introductory psychology class 
expressing interest in an experiment that involved receiving 
career guidance services participated in the study. All students 
received partial course credit for participating in the study. 
Table 1 contains a summary of student demographic data for each 
treatment condition. The mean age of students was 18.8 years 
(SD-l.7) with 75.0% being female and 67.2% white, 15.6% black, 
12.5% Native American, 4.7% Asian American. The majority of 
students were freshmen (59.4%) and the three most popular 
declared majors were business (39.1%), psychology (9.4%), and 
clothing and textiles (6.3%); 10.9% were undecided majors. Some 
students reported prior career services experience: individual 
counseling, 21.9%; career course, 18.8%; and some type of CACG 
system, 14.1%. It was assumed that the sample sufficiently 
approximated typical students served in a college career center 
in order to permit a reasonable degree of external 
genaralizability of the results. This was based on the fact 
that: (1) the study was described to potential subjects as 
involving career guidance services, (2) students in this study 
freely elected this particular experiment among numerous 
alternatives, (3) the demographic characteristics of the sample 
were very similar to actual career center clientele, and (4) 
about 15 students referred friends, roommates, etc. to the career 
center during the study. 

Instrumentation 

The four types of instruments used in this study included: 
1) general measures of career development; 2) specific measures 
of students' perceptions of the use of SIGI and SIGI PLUS; 
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3) measures used to verify the nature of the experimental 
treatment; and 4) an instrument for collecting demographic data. 

The Occupational Alternatives Question (OAQ) ( Zener & 
Schnuelle, 1972; modified by Slaney, 1978; 1980) measures level 
of career decidedness. Test-retest reliability for the OAQ was 
reported at .93 (Redmond, 1973) and found to be stable over a six 
week period (Slaney, 1978) . Concurrent validity was demonstrated 
by Slaney, Stafford, and Russell (1981) . 

My Vocatio nal Situation (MVS) (Holland, Daiger, & Power, 
1980a) measures vocational identity, the perceived need for 
information, and perceived barriers to career decision making. 
Holland, Daiger, and Power (1980b) presented scale reliabilities 
(KR 20) ranging from .23 to .86, with the Identity Scale 
demonstrating the highest degree of internal consistency. 
Construct validity for the MVS was demonstrated by Holland, 
Daiger, and Power (1980b). 

The SIS1 and SIGI PLUS Evaluation Forms (Peterson, Sampson & 
Reardon, 1984) (see Appendix A) . These two instruments are 
identical with the exception that the name SIGI or SIGI PLUS was 
used throughout each respective instrument. Each instrument had 
3 scales; one scale measuring the effectiveness of the computer 
in helping the user to become familiar with oneself and the world 
of work (Analysis) ; one scale measuring the effectiveness of the 
CACGS in developing and evaluating career options (Synthesis) ; 
and one scale measuring global impressions and human factors 
(user friendliness) dimensions (Computer Effect) . The 
intercorrelations among the three scales ranged from .39 to .60 
while the respective alpha reliabilities were Analysis .83; 
Synthesis, .77; and Computer Effect, .87 (Peterson, Ryan-Jones, 
Sampson, Reardon, & Shahnasarian, 1987) . 

The Comparative Computer Rating Form (Reardon, Peterson & 
Sampson, 1985) (see Appendix B) contained items identical to the 
SJfil and SIGI PLUS Evaluation Forms , with the exception that 
instead of rating the degree of student agreement with various 
statements about SIGI or SIGI PLUS, students indicated on a 
bi-polar 7-point scale whether SIGI or SIGI PLUS more accurately 
represented various statements, after they had used both systems. 

The SIGI Progress Record, and the SIGI PLUS Progress Record 
(Reardon, 1984a) (see Appendices C & D) were designed to verify 
the extent to which SIGI and SIGI PLUS were actually used by 
students. Basic demographic data and information related to 
students' prior experience with career counseling services, 
including computer applications, were also collected (Reardon, 
1984b) (see Appendix E) . 

PrpcfflMrfff 

The 64 students were randomly assigned to first use either 
SIGI (Educational Testing Service, 1984) (n*32), or SIGI PLUS 
(Educational Testing Service, 1985b) (n*32) and then use the 



6 



other system ten days later. The study was conducted at a 
university career resource center, students In the SIGI and SZGI 
PLUS groups were encouraged to use supplemental print-based and 
audio-visual career information materials as part of the 
treatment and were asked not to use other available CACG systems 
in the center until the data collection was completed. Students 
using SXGI were asked to complete all five subsystems, while 
students using SIGX PLUS were instructed to complete specific 
sections according to their individual needs. Using the systems 
in this manner reflected recommendations for client use presented 
in the Counselor's Handboo k for SIGI on Microcomputer (Chapman & 
Seibel, 1982) and the SIGI PLUS Counselor's Manual (Educational 
Testing Service, 1985a) . 

All students attended a group specific orientation meeting 
(see Appendix F) where: 1) an overview of the study was provided; 

2) a research participation release form (see Appendix G) , a 
demographic questionnaire, the OAO and tha MVS were completed; 

3) an introduction to the purpose, operation, and procedures 
associated with SXGI or SIGX PLUS was provided; 4) an explanation 
of data collection procedures was presented; 5) initial 
appointments were scheduled for SIGX or SIGI PLUS, and 6) a tour 
of the career resource center, including the location of relevant 
resources, was completed. Both groups were encouraged to ask 
questions, obtain feedback, and seek support from available staff 
members during the study. All students completed SIGI or SIGI 
PLUS within a 10 day period at which time the following 
instruments ware completed: 1) MVS ; 2) £&£; 3) SIGI Evaluation 
Form or SIGI PLUS Evaluation Form ; and 4) SIGI Progress Record or 
SIGI PLUS Progress Record . Students; then scheduled their use of 
the second system and completed the second system within a 10 day 
period. The Comparative Comput er Rating Form was completed at a 
follow-up session, and a debriefing of the purpose of the study 
was then provided to all students. 

Pflta ggfliag ftftfl ABtlYPiP 

For data analysis, the OAQ score values were receded (1) low 
and (2) high career decidedness based on the median split of the 
scores of the students. High career decidedness included those 
individuals who indicated either a first choice only o_r_ a first 
choice plus alternatives. Low career decidedness included 
students who listed alternatives, but no first choice, as well as 
those who had neither a first choice nor alternatives. 

Students' scores on the Vocational Identity subscale of My. 
Vocational Situation were divided into two levels of vocational 
identity based on the median split of the current sample. Thus 
students scoring nine or less were regarded as having low 
vocational identity (1), and those scoring ten to eighteen points 
had high vocational identity (2) . 

The sum of the "N N responses to the four items on the 
Information Needs subscale (MVSIN) of the M y Vocatipnal situation 
provided an index of students' expressed information needs. For 
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analyses, students were divided into those who (1) expressed no 
need for information, and (2) those who expressed current needs 
for information* Similarly, the Barriers subsoale of the MVS 
(MVS BAR) , provided an index of students' perceived barriers in 
obtaining their career goals. Students wers. split into two 
groups, those with (l) no barriers, and (2) those who faced 
barriers to accomplishing their goals, students' year in school 
(YEAR) was obtained from the demographic questionnaire and 
recoded for analysis into (1) Freshman, or (2) Sophomore, Junior, 
Senior, or Special student (adults not enrolled full time) . 

A multivariate analysis of variance (MANOVA) using wilk's 
lambda criterion was planned to examine the effects of treatment 
on students' perceptions of the CACG system used. The client 
attributes of career decidedness, vocational identity, 
information needs, and barriers were measured by the OAQ and the 
MVS . Year in school was also examined in the analyses on the 
dependent variables of Analysis, Synthesis, and Effect. For 
statistically significant findings, post hoc analyses involved 
the use of univariate ANOVAs. 

students' responses to the Comparative Compute r Rating Form 

were analyzed according to the functions of Analysis, Synthesis, 
and Effect. Examination of individual items was planned to 
obtain a better understanding of the students comparison of SIGI 
and SI6I PLUS. Single-sample t-tests were used to determine if 
differences in users' preference for a particular CACG system 
were significant. Finally, a large-sample binomial probability 
test was planned to determine if the students' choice of a single 
preferred CACG system was significant. For all analyses, alpha 
was set at .05. 

Results 

Intercorrelation Matrix and the Effects of T i me in Treatment 

Table 2 presents the zero order correlations among the 
variables in this study. For these 64 students, initial 
vocational identity (MVSID1) and information needs (MVSINl) were 
negatively correlated, as were post-treatment vocational identity 
(MVSID2) and information needs (MVFTN2) . Thus as students' 
vocational identity scores increased, their information needs 
decreased. For all measures of student attributes, ( OAQ . MYSID, 
MVSIN, MVS BAR) , students' pre- and post-treatment scores were 
positively correlated, demonstrating relative stability over the 
10 day period, despite the intervening CACGS treatment. There 
was a negative correlation between post-treatment vocational 
identity (KVSID2) and post-treatment perceived barriers 
(MVSBAR2) , and a positive correlation between post-treatment 
barriers (MVSBAR2) and information needs (MVSIN2) , but the same 
relationship did not exist prior to treatment. After using a 
single CACG system, students' higher vocational identity was 
related to decreased barriers, and decreased barriers were, oddly 
enough, positively correlated to increased information needs. 
Intercorrelations with the Comparative Computer Rating Form 
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dependent variables, (CAN, CSYN, CEF, and PREFSYS) , are less 
interpretable . 

The tine spent using the first CACGS (TIME1) was positively 
correlated to time spent on the second CACGS (T1ME2) , implying 
that students who spent more time using the first system, also 
spent more time using the second CACGS. Students who used SIGI 
initially spent significantly more time using the system than 
those using SIGI PLUS [F(l,59)«6.33, p-.Ol]. The amount of time 
spent using a CACG system was not related to the client attribute 
variables of career decidedness, vocational identity, information 
needs, or perceived barriers. 

Evaluation of SIOI and SIQI PLUS 

The results of a multivariate analysis of varience with the 
three dependent variables, Analysis, Synthesis, and Effect, found 
no significant differences (p«.52) between the CACG systems (see 
Table 3). Users rated SIGI and SIGI PLUS equally positive for 
their effectiveness in obtaining self and occupational 
information (Analysis) , viable career options (Synthesis) , and in 
attractiveness of interacting with the computer (Effect) . 

In order to ascertain whether the effectiveness of the CACG 
system was a function of the students' level of career 
decidedness (QAQ) , vocational identity (MVSID) , information needs 
(KVSIN) , perceived barriers (MVS BAR) , or year in school (YEAR) , a 
series of 2 X 2 (treatment X level of attribute) MANOVAs using 
high and low levels of the various attributes were performed. 
There were no main effects nor interaction effects of YEAR, 
MVS-Identity or MVS-Barriers on the three dependent variables of 
Analysis, Synthesis, or Effect. However, the MANOVA indicated a 
significant main effect of MVS-Information Needs, which was 
further investigated using a 2 X 2 ANOVA. Students with high 
initial information needs rated both CACG systems significantly 
higher (p < .05) on the Analysis scale than those without initial 
information needs, (see Table 4, Figure 1). Students with 
information needs found both SIGI and SIGI PLUS more effective in 
helping them to obtain information about themselves and the world 
of work than did persons without needs for information r 

Students' level of vocational identity (MVSID) increased 
significantly [F(l,6l)*7.95j p~.006} as a result of using SIGI 
and SIGI PLUS (Table 6) . Examination of the pre- and post-CACG 
treatment cell means on the MVS-Identity subscale across the two 
CACG systems shows that while both systems resulted in higher 
identity scores, SIGI PLUS users showed a greater increase in raw 
scores (SI^I; +.68; SIGI PLUS: +2.13), and less variability among 
users (see Table 5) . On both systems, students with initial high 
levels tended to retain a high level of vocational identity after 
using a CACG system. Students with an initial low vocational 
identity score were more likely to increase their vocational 
identity score if they used SIGI PLUS, than if they used SIGI 
(see Table 6) . 
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Repeated measures MANOVA indicated a significant change in 
students' information needs (MVSIN) as a result of using the CACG 
systems [F(l,60)-8.88, p-.004] (Table 7). Inspection of cell 
means (see Table 5) shows that users of both SIGX and SIGI PLUS 
decreased their raw score on need for information (MVSIN) as a 
result of using the CACGS (SIGI; -.51; SIGI PLUS: -.74). Table 7 
shows the distribution of students on pre- and post-treatment 
MVS -Information Needs subscale by the CACG system used. SIGI and 
SIGI PLUS appear equally capable of satisfying users' needs for 
information . 

The students' level of career dec.idedness (Q&Q) did not 
change significantly as a result of using the CACG systems. 
Likewise, there was no significant pre-post change in the number 
of perceived barriers (MVSBAR) . 

comparison of 8101 and SIQI PLUS 

After using boLh CACG systems, students compared the two 
systems on the Comparative Comp uter Rating Form, rating their 
effectiveness on the Analysis, Synthesis, and Effect functions of 
CACGS. Single sample t-tests were used as a conservative measure 
to determine if the preference for either system was 
statistically significant. The students found SIGI and SIGI PLUS 
equally Ivalpful for obtaining information about themselves and 
the world r* work (CAN: mean—. 04, SD=*.87, t— .001, p>.05). The 
students fex*. that SIGI and SIGI PLUS were equally effective in 
suggesting viable career options (CSYN: mean*. 05, SD*»1.18, t— 
.003, p>.05). The students rated SIGI and SIGI PLUS similarly on 
the attractiveness of using the computer (Effect) (CEF: mean=».08, 
SD*1.05, t«-.0008, p>.05). 

The final item on the Com parati ve Computer Rating Form asked 
the students to indicate which system they would prefer if forced 
to make a choice. Of the 63 students who responded after using 
both CACG systems, 24 or 38% preferred SIGI, while 39 or 62% 
preferred SIGI PLUS. A larg* sample binomial probability test 
indicated that while the majority of student preferred SIGI PLUS 
overall, the significance was not at the .05 level (z*.19, 
P-.06). However, we can be 94% confident that users in this 
study, who experienced both CACG systems, preferred SIGI PLUS to 
SIGI. Table 8 contains the means and standard deviation of 
students' responses to each item on the Comparative Computer 
Rating Term. 

The impact of students' initial career attributes on their 
preference for either SIGI or SIGI PLUS for the Analysis, 
Synthesis and Effect functions was examined through a series of 
one-way ANOVAs. Pre-treatment levels of career decidedness, 
vocational identity, information needs, perceived barriers, and 
the students' year in school had no significant impact on their 
rating of the two CACG systems. 

Similarly, the impact of students' levels of career 
attributes subsequent to their first CACGs treatment, was 
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examined. Students with perceived barriers (MVSBAR2) after their 
use of either SIGI or SIGI PLUS indicated a significantly higher 
rating for SI6I in helping them to obtain and consider viable 
career options (Synthesis) [F(l,57)*4.79, p-.03] (see Table 9). 

Neither pre- nor post-treatment levels of career 
decidedness, vocational identity, information needs, perceived 
barriers, nor year in school had any significant impact on 
students' final choice (PREFSYS) between SIGI and SIGI PLUS. 

Discussion 

Differential iapiet of SIfll and SIGI PLUS 

The use of SIGI PLUS resulted in greater gains in vocational 
identity, especially for those persons with low pre-treatment 
identity scores. Users of both systems also tended to express an 
overall preference for SIGI PLUS over SIGI. These findings 
suggest that the changes in system content and process in the 
evolution from SIGI to SIGI PLUS had a direct impact on system 
effectiveness. In particular, making the system more inclusive 
in the assessment options and information available ( content ) and 
more flexible and user friendly in terms of color displays, 
layering of information, and improved graphics ( process ) has 
resulted in improved system effectiveness. 

The amount of time a person spends using a CACG system 
appears to be affected by the characteristics of both the 
individual and the system . Individuals who were very thorough in 
their use of one system demonstrated similar behavior with the 
second system used. Persons spent more time using SIGI than SIGI 
PLUS, indicating that the flexibility (user control of system 
functioning) incorporated into SIGI PLUS resulted in lower mean 
time on the system. This is an important finding in terms of 
cost-effectiveness. It appears that SIGI PLUS is more cost- 
effective than SIGI because more persons would tend use SIGI PLUS 
in a given period of time in comparison to SIGI, and because SIGI 
PLUS was more effective than SIGI in improving individuals' 
vocational identity. 

In this study, SIGI and SIGI PLUS were equally effective in 
assisting individuals in obtaining needed information, as shown 
by MVSIN scores. Individuals perceived SIGI and SIGI PLUS as 
equally effective in? 1) helping them increase self and 
occupational information, 2) developing and evaluating career 
options, and 3) ease of system use. This would imply that the 
changes in system content and process in the evolution from SIGI 
to SIGI PLUS had no appreciable impact on these specific 
outcomes . 

Finally, additional support for Holland's theory of 
vocational choice (Holland, 1985) is provided by virtue of the 
negative correlations between: 1) vocational identity, ar.d 2) 
need for information and barriers, e.g. as both the need for 
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information and the indications of barriers decreased, vocational 
identity increased. 

BcralYlXfBgt of fi&gg gYftfM 

Data from this particular study indicate that it is not 
appropriate to make generalizations about the equivalence of 
different versions o£ a CACG system when substantial content and 
process changes have been made. For example, SIGX PLUS cannot be 
considered equivalent to the original SX6I system in view of the 
results obtained in this study. This lack of equivalence is a 
function of differences in system content (expanded self- 
assessment options, material for adults, and job search 
information) and system process (flexibility and user 
friendliness) . 

Implications for Practice 

It would appear that the client perceived need for 
information is a valid prescreening variable for identifying 
individuals who are most likely to be appropriate users of a CACG 
system. Also, in evaluating a CACG system that has evolved 
substantially over time, practitioners need to be careful in 
making assumptions that the validity of the prior system version 
transfers to a revised version. Features that were or were not 
effective in one version of a CACG system may not be retained in 
revised versions. In situations where CACG system content or 
process changes have occurred, practitioners need to evaluate the 
issue of equivalency either through existing research, or at a 
minimum, conducting a thorough comparison of system features. 

Implications for System Design 

System developers need to pay particular attention to the 
breadth of content included in CACG systems. Many individuals 
seeking career guidance have a specific need for information and 
a CACG system with a diversity of information available will be 
more likely to meet individual ne^ds. Paying close attention to 
flexibility and user friendliness is also important in helping 
users obtain full benefit from the information included in a CACG 
system. Developers also need to fully explain and document the 
scope and purpose of system changes leading to revised versions 
when they are first introduced. Preliminary comparative studies 
between new and old versions should be reported prior to 
introducing new software. This will help all users make better 
transitions to revised versions of CACG systems. New system 
versions also require new training and marketing efforts by 
developers . 

Implications for Future Research 

The amount of time an individual spends using a CACG system 
appears to be influenced by both individual and CACG system 
characteristics. The undergraduate students in this study spent 
less time using SIGX PLUS compared to SIGX, but adult career 
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changers night be expected to spend more time using SIGX PLUS. 
This might occur both because of user characteristics and changes 
intentionally designed into SIGI PLUS. It would be important to 
more fully understand the impact of the amount of time spent 
using a CACG system on various career guidance outcomes. It 
would also be important to replicate the system comparison 
methodology used in this study with different uner groups and 
different system versions in order to further validate this 
approach for determining the equivalency of different versions of 
a CACG system. 
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Table 1 

Group and Aggregate Demographic Characteristics of the Sample 

Standard 

characteristic Percentage Mean — . — PeviaUsn 

Age 18.8 1.7 

Freshmen 59.4% 

Males 25.0% 
Females 75.0% 

White 67.2% 

Native American 12.5% 

Black 15.6% 

Asian 4.7% 

Business Majors 39.1% 

Psychology 9.4% 

Clothing & Textile 6.3% 

Undecided 10.9% 

Individual Counseling 21.9% 
Career Course 18.8% 
CACG System 14.1% 
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Table 2 

Intercorrelation Matrix (n«64) 



Correlations 

Variables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 



1. Analysis 












2. Synthesis 


.40*** 










3. Effect 


.58*** 


.69*** 








4. ¥ear a 


-.07 - 


.11 - 


.20 






5. Timel 


.08 - 


.15 


. 05 


. 16 




6. OAQl C 


.04 


.09 


.03 


-.14 


-.10 


7. MVSIDl d 


.02 - 


.08 - 


.08 


.13 


-.11 


8. KVSINl 6 


.21 - 


• 2 X 


.02 


-.08 


.14 


9. MVSBARl f 


-.15 - 


.04 - 


.14 


.07 


.17 


10.OAQ2 b 


-.06 


.10 


.15 


-.18 


-.26* 


ll.HVSID 2c 


-.02 


.13 


.18 


.05 


-.35** 


12.MVSIN2 d 


.28* 


.05 - 


.07 


-.05 


.21 


13.MVSBAR2 e 


-.06 - 


.02 - 


.21 


-.02 


.20 



.10 

.18 -.24* 

.00 -.08 -.07 

.42***. 15 .13 -.02 

.15 .58***-. 31**-. 23* .37*** - 

.05 -.34** .47***. 05 -.11 -.50*** - 

.07 -.11 .05 .62***. 01 -.32 .24* 
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14 Tim«2^ > 


.16 

• lA w 


.04 


.14 


.09 


.25* 


. 14 


.13 


-.00 


.21 


.11 


.09 


-.03 


.21 






15 PREFSYS^ 


.00 


-.13 


-.04 


-.01 


-.02 


-.09 


-.01 


.09 


-.14 


.10 


-.06 


.04 


-.14 


-.13 




16.Comp.- 


-.08 


-.21 


-.10 


-.20 


-.09 


-.01 


-.10 


.22* 


-.03 


.16 


-.17 


.10 


-.00 


-.32** 


.66*** - 


Analysis 

17.Corop.- 


-.15 


-.22* 


-.05 


.00 


.10 


-.14 


-.13 


.14 


-.18 


.08 


-.03 


.06 


-.28* 


-.18 


.78***. 67*** - 


Synthesis 
18.Comp,~ 


-.20 


-.26* 


-.08 


.09 


.09 


-.13 


-.05 


.13 


-.02 


.08 


-.10 


.08 


.01 


-.15 


.69***. 65***. 74*** 



Effect 



* Year in school ( 1- Freshman, 2 -Sophomore, Jumnior, Senior, Special Student) 
b Time on CACG system (1» less than 100 minutes, 2«more than 100 minuts) 

c occupational Alternatives Questionnaire Scale (1-first choice only or first choice plus alternatives, 2-alternativee only 
nor alternatives) 

d My Vocational Situation - Identity Scale (1-low identity, 2»high identity) 

• My Vocational Situation - Information Needs Score (1-no information needs, 2«need information) 
f My Vocational Situation - Barriers Scale (l«no barriers, 2»barriers) 
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Table a 

manova Summary Table 



SIGI (n»30) SIGI PLUS (n»3i) Sign. 
M SD M SD F(l,60) of F 



Dependent 

Variables 
Analysis* 
Synthesis* 
Effect* 


.74 
.40 
.37 


.50 
.82 
.66 


.77 
.74 
.48 


.53 

.55 3. 
.54 


04 .84 
07 .06 
47 .50 




Multivariate Tests of 


Significance 




Test Name 


Value 


F 




Error DF 


Sign, of F 


Pillais 
Hotellings 
W i Dc ' s 
Roys 


.04 
.04 
.10 
.04 


.76 
.76 
.76 


3.0 
3.0 
3.0 


54.0 
54.0 
54.0 


.52 
.52 
.52 
.52 



a Scoringi 5-point Likert-type scale, where -2 - strongly disagree; -1 « 
disagree; 0 « neutral; +1 -agree; +2 « strongly agree 
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Table 4 

Comparison of the Analysis Function A ccording to CACG 
Systems w ith Mv Vo cational Si tuation Information Needs Scale 
as Moderator Variable 



Systems 

MVSIN SIGI SIGI PLUS 

mS. sn M SI 



HIGH . 84 




43 


.97 .44 






(n-16) ' 




(n=16) 




LOW .68 


• 


58 


.57 .60 






(n=12) 




(n=13) 




Wilk's 


Lambda 


Multivariate Tests of Significance 


Test Name 


Value 


F 


DF Error DF 


Sign, of F 


CACG System 


.96 


.73 


3.0 51.0 


.54 


MVSIN 


.82 


3.76 


3.0 51.0 


.02* 


System 










X MVSIN 


.97 


.45 


3.0 51.0 


.72 



Univariate Source Table 



Source of 
Variation 


ss 


MS 


DF 


F 


Sign, of 
F 


Main Effects 


1.10 


.55 


2 


2.11 


.13 


System 


.00 


.00 


1 


.03 


.86 


MVSIN 


1.10 


1.10 


1 


4.20 


.05 


System 










.37 


X MVSIN 


.21 


.21 


1 


.81 


Explained 


1.31 


.44 


3 


1.67 


.18* 


Residual 


13.86 


.26 


53 







On a 5 -point Likert-type scale scored as follows: ~2 * strongly 



disagree, -1 * disagree, 0 * neutral, +1 * agree, and *2 * strongly 
agree* 

* p < .05 
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Table 5 

Raw Score Mean Comp arison of Client Attributes Pre- and Post 
Treatment across CACG Systems 



Client SIGI SIGI PLUS Combined Systems 





n 


oil 




M 






9 AT 
4 i Hi 


• DO 


? 11 47 


2 36 


. 58 


0AQ2 


2.38 


.49 


2.16 .45 


2.26 


.48 


MVSIDl b 


(n- 


29) 


(n«32) 


(n«61) 




9.16 


4.74 


9.78 4.72 


9. 48 4 


.70 


MVSID2 


9.84 


5.32 


11.91 4.16 


10.89 4 


.84 




(n= 


=31) 


(n*31) 


(n-62) 




MVSIN1 C 


.55 


.89 


.58 .76 


.56 


.82 


MVSIN2 


1.06 


X * 18 


1.32 1.33 


1.19 1 


.25 


MVSBARl* 


(n= 


=31) 


(n=31) 


(n==62) 




3.42 


.88 


3.58 .67 


3.50 


.78 


MVSBAR2 


3.29 


.94 


3.58 .72 


3.44 


.84 




(n* 


=31) 


(n=31) 


(n=62) 





"1" following the client attribute refers to pre-treatment data. 
"2" following the client attribute refer* to post-treatment data. 



Scorings 1» first choice only, no alternatives? 2» first choice with 
alternatives; 3« no first choice, alternatives only; 4- neither first nor 
alternatives. 

Scoring: total number of "false" responses on the Vocational Identity 
subscale of the MVS. 

Scoring: total number of "no" responses on the Informacion Needs subscale 
of the MVS. 

Scoring: total numbev of responses on the Barriers subscale of the MVS. 
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Table 6 

Repeated MANOVA Summary Tab le - Change in Vocational 
Identity as a resu lt of using the CACGS. 



SIGI 
MVSID2 



SIGI PLUS 
MVSID2 



HX 


LO 






HI 


LO 




HI 






HI 








n=15 


n=o 


15 




n=14 


n=2 


16 


MVSIDl 






MVSIDl 








n=3 


n=l3 


16 




n=9 


n=6 


15 


LO 






LO 








18 


13 


31 




23 


8 


31 



Source 



Repeated Multivariate Source Table 

SS DF MS F 



Sign 
of F 



Within Cells 23.17 
System .37 



61.0 
1.0 



.38 
.37 



.97 



.33 



Within Cells 
Trial* 


6.09 
.79 


61.0 
1.0 


.10 
.79 


7.95 .00** 


System 
X Trial 


.12 


1.0 


.12 


1.17 .28 


Total 


7.00 


63.0 


• XX 





Tra__ „_ 

Vocational Identity 

** p < .01 
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Table 7 

Repeated MANQVA Summary Tab le = Change in Information Needs 
as a result of using the CACGS. 



SX6I 
MVSIN2 



SIGI PLUS 
MVSIN2 



HI 



LO 



HI 



n=12 



n=7 



19 



MVSIN1 



HI 

MVSIN1 



HI 



n=ll 



1*0 



n=7 



8 





n-10 


12 


n=2 n-11 


1 T 


LO 






LO 




14 


17 


31 


13 18 


31 




Repeated Multivariate Source Table 












Sign 


Source 


SS 


DF 


MS F 


of F 


Within Cells 


21.97 


60.0 


.37 




System 


.03 


1.0 


.03 .09 


.77 


Within Cells 


7.84 


60.0 


.13 




Trial* 


1.16 


1.0 


1.16 8.89 


.00** 


System 










X Trial 


0 


1.0 


0 0 


1.00 


Total 


9.00 


62.0 


.15 





* Trial is the change between pre- and post-treatmer MVS- 
Information Needs 



** 



p < .01 
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Table 8 

Means and Standard Deviation! of fl-tems on Comparative Computer Rating Form 

COMPARATIVE COMPUTER RATING FORM 
by 

Gary w. Peterson, Ph.D. , Rebecca E. Ryan-Jones, M.Ed. , 
James P. Sampson, Jr., Ph.D. and Robert C Reardon, Ph.D. 



Directions s Listed below are items which contain the word pairs SIOI and SIGI+ at either 
end of the scale and seven spaces between the pairs. Please rate SIGI and SIGI PLUS on 
each item according to the following guideline. 



SIGI 



Very Quite Only 

Strongly Strongly Slightly 
SIGI SIGI SIGI 



Equally 
SIGI & 
SIGI + 



SIGI-*- 

Only 

Slightly 
SIGI+ 



Quite 

Strongly 
SIGI ♦ 



Very 
Strongly 
SIGI+ 



1. The computer helped me to learn SIGI t t 

much more about several occupations. (A) 



2. The computer was helpful in 



si£2 L 



showing me whether I needed more information 
about occupations before making career 
decisions. (A) 



5. The computer helped me to 



SIGI 



understand the rewards potential occupations 
offer, such as salary, interesting work, 
prestige, variety, and challenge. (A) 



SIG I 



8. The computer helped me feel 

confident that I would find most of the 
final list of potential occupations 
satisfying. (S) 



SIGI L 



I -05* 



J f SI9I+ 



SD 1^7 
SD 1.7 



SIGI ; i 



3. Using the computer was like 

talking to a career counselor. (E) 



4. The computer presented logical SIGI : t-.34; 

career options given my values, interests, 
and abilities. (S) 



t-.03t 



6. I felt the computer understood my 
career problems. (E) 



7. I have learned about soma new SIGI t t 

educational programs as a result of using 
the computer. (E) 



t-.14t 



I tP8; 



» : SIGI+ SD ia 



* SIGI+ SD 



J $TG* + sd 2u3. 



J 5 SJGJ+ 



SD iJJ 
SD 1.1 



» * SIGH- SD Xj£ 



SIGI 



Very 



Quite 



Only 



Equally 



_SIGI+ 
Only 



Quite 



Very 



0 

ERJC 
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Strongly Strongly Slightly SIGI & Slightly Strongly Strongly 
SIGZ SIGI SIGI SIGI* SIGI+ SIGI + SIGI+ 

9. The computer satisfied me with SIGI i i : : .11: t SIGI+ SD 

the variety of career options it gave me ' o 
consider ♦ (S) 

10. The computer helped me to become SIGI t t s s .40; i SIGI+ SD Ua 

more familiar with the educational 
requirements of potential occupational 
choices, (h) 

XX. The computer was helpful in SIGI t s-.89s i ? s SIGI+ SD Ui 

accurately clarifying my values. (A) 

12. The computer helped me to feel SIGI t t t i .31% * SIGI* SD Xx& 
more hopeful of finding a satisfying 

occupation* (S) 

13. I can seriously consider most of SIGI t : s ,0 0? : s SI0I+ SD JU4 
the occupations the computer suggested. (S) 

14. My family or friends would like SIGI x : :-,03; 5 ; £I£1± SD 

the outcomes suggested by the computer* (E) 

15. The computer satisfied me with SIGI ; ; ; »02;_ t : SIGH- SD 1/7 
the number of career options it gave me to 

consider. (S) 

16. The computer was helpful in SIGI ; :-.21s t t t SIGI+ SD JU2 
accurately clarifying ray interests. (A) 

17. The computer was helpful in SIGI ; :~.14; : % 1 SIGI+ SD 1/7 
showing me whether I needed more information 

about myself before making career decisions. 
(A) 

18. The computer helped me understand SIGI s :-.44: : t : SIGI+ SD i^I 
the demands associated with potential 

occupational choices, such as amount of free 
time, vacations, and continuing education. 
(A) 



9 
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SIGZ i I t s_ : t SIGI+ 

Vary Quit* Only Equally Only Quit* Very 

Strongly Strongly Slightly SIOI & Slightly Strongly strongly 

SIGI SIOI SIOI SIGI+ SIGI+ SIOI * SIGX+ 

19. The computer answered most of my SIGI : s t s .26: s SIQI+ SD X±5. 

career questions to my satisfaction. <S) 

20. The computer helped me to SIOI : t : i .50: t SIGH- SD i^fe 

identify important milestones to achieve in 

attaining a career, such as educational 
degrees, training, or licenses. (A) 

21. The computer helped me better SIGI t t t t .23: s SIGI+ SD XJL 

understand how the world of work is 
organised. (A) 

22. I understand myself better now. <E} SIGI : :-.llt : : i SISIt SD X*A 

23. I felt better about my career SIQI : : s i .30: : SIGI+ SD j t § 
after I used the computer. 

24. The computer helped me become SIGI : : t ; .13: : SIGH- SD 1. 7 
more confident of being able to choose a 
satisfying occupation. (E) 

25. If I had to make a choice, I SIGI U§ SISI+ SD P-*l 

would prefer (Circle either SIGI or 
SIGI+) tSIGI - 1; SIGX+ - 2Ji 

(c) Copyright 1987. All rights reserved. 



SIGI : 


■J.,- i 1 m 


s — uuu — 

: : 


I SIGJ+ 


SIQI, f 


t f 


s .30: 


: SIGI+ 


SIGI : 




: tJ3: 


: SIGI* 


SIGI 




i,« 


SIOJ+ 
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Table 9 

Impact of Perceived Barriers Subsequent to using the first 
CACG System on the Comparison of SIGI and SIGI PLUS on the 
Synthesis Function of the System. 



MVSBAR2 

HIGH LOW 

Comparison on 

Synthesis MEAN* -.47 .21 

(n=22) (n=37) 



Univariate Source Table 



Source of 
Variation 


SS 


MS 


DF 


F 


Sign, of 
F 


Main Effects 


6.34 


6.34 


1 


4.79 


.03* 


MVSBAR 


6.34 


6.34 


1 


4.79 


.03 


Explained 


6.34 


6.34 


1 


4.79 


.03* 


Residual 


75.44 


1.32 


57 






Total 


81.79 


1.41 


58 







On the Comparative Computer Rating Form , a negative number indicate* a 
preference for SIGI, while a positive number indicates a preference 



for SIGI PLUS, 
p < .05 
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